Early estimating using COSMIC-FFP

One of the prerequisites for introducing the COSMIC-FFP functional size measurement in organisations is the quality that it is suitable for early estimation. This paper will explain that the early estimation techniques using COSMIC-FFP can be used in the early stages of software development or maintenance. Two early estimation techniques have been developed based on COSMIC-FFP: approximate COSMIC-FFP and the refined approximate technique. The use of these two techniques have been investigated in four different commercial sectors. The investigated early size estimation techniques appear to be environment dependent. This paper will explain how the values for these figures can be derived. Furthermore the precision of these techniques have been investigated to establish that they give a good estimate, compared to a detailed size measurement in a later stage of the software development. Finally the use in early cost estimation is described. 1. The need for (early) estimating Estimates help development organizations make good business decisions. They help determine the feasability of completing projects within the time, cost, and functionality constraints imposed by customers and the organization's own resources [1]. They help to avoid developing software with a low probability of meeting the organization's goals. Estimates can also help to set expectations about achievable development schedules in terms of time and cost. Time and cost are usually important figures in a business case and to most people, estimation is usualy associated with time or cost rather than with size. Time and cost estimates are functionally dependent on effort estimates.Effort estimation consists of two elemants: the project size in some quantifiable unit and the rate at which an organization can deliver one of those quantifiable units. Thus, projects cannot be accurately estimated consistently without a gauge on size [2]. The main part of this article will deal with size estimation. Estimates are made throughout the complete lifecycle of an information system, from before the development begins, through the development process, during the transition of the system to the customer and while the software is being maintained [3]. The most important estimates are made in the early stages of the lifecycle when the decision has to be made whether development of an information system is possible within the constraints of the business case [4]. In those early stages little detail is known, but the impact of the decision is large. An estimate of the size of the software to be developed has to be made with as little information as possible and as much accuracy as possible. There are a number of techniques to do this already (see section 2). This paper will deal with two early size estimating techniques based on COSMIC-FFP. 2. Early size estimating Techniques for early size estimating are derived from experience about what aspects of an information system can predict size in an early stage of development. A number of techniques have been developed, including: Fuzzy logic is a way of systematizing comparisons with past work. It was first introduced by Zadeh [5] and adapted to the field of software sizing by Putnam and Myers [6]. Previously built systems are subdivided into six categories based on size. Within each size category there are four ranges representing quartiles. Comparing a new information system with projects done in the past yields an estimate of the size of this system and a probability range. This method is also a subjective methode because it relies on experts to compare a new project to projects done in the past. Standard-component sizing identifies a number of key components which can be compared to historical data about such components [6]. Based on the number of needed key components, the average and extreme (low and high) size for each components a size estimate can be produced. This method relies on data of previous projects. Proxy-based Estimating assigns a size range to each conceptual method within the software to be sized using historical data for such methods [7]. With the ProBE method a prediction interval for a desired accuracy can be calculated. This method too relies on data of previous projects. Wideband-Delphi, where a number of experts individually estimate the size of a new information system based on the requirements [8]. A moderator calculates the average size estimate and returns it with all the other estimates to the experts for reestimation. This process continues until the estimates are close enough together. It is a time-consuming method which relies on a number of experts within the organization. FPAi, contains a number of approaches to estimate the functional size [4]. Next to some form of fuzzy logic and standard-component sizing it also contains an approach that can be seen as a compromise between the Delphi method and Early & Quick. In this approach a number of estimators establish the weight of a number of requirements. To each weight a minimum, a most likely and a maximum estimate have been assigned. The approach is less sophisticated than E&Q, but less time-consuming than Delphi. Early and Quick Function Point Analysis provides estimators with an early forecast of the functional size that can be used for preliminary managerial decisions [9]. The method is based on identifying software objects at different levels of detail. For each type of object there is a minimum, a most likely and a maximum estimate where the degree of uncertainty is higher for more superficial objects. This method is fairly objective because it only relies on the skill of the estimator to correctly identify and assess the software objects, which should be objective to a large extent. The original method was based on IFPUG function points. Later the method has been adapted to express the functional size based on the COSMIC-FFP method [10]. Depending on the type of software that has to be produced and the environment in which it has to be developed all of these methods can be useful, but almost all of these methods rely on expert experience and knowledge about past projects and thus are in some way subjective. Another way of making early size estimates is by simplifying detailed objective size estimation techniques. 2.1. Early size estimating using function points Several early estimating techniques have been derived from function point analysis [11]: The first simplification is when you do know the detailed elements, but do not know exactly their size: take the average value for the EI, EO and EQ (because usually there are as much low as high complexity functions) and take the low complexity for ILF and EIF (because more than 90% of the files will have this complexity). This simplification – also known as FPS (Function Points Simplified) – can have an accuracy range within 5% [12]. A further simplification is the fact that there is usually a linear relation between the number of logical files and the number of functions working on that files. So if there is already a data model available in third normal form factors can be used to estimate the expected number of function points associated with the software based on this data model. For each ILF entity type 25 function points are counted and for each EIF entity type 10 function points are counted. If only a conceptual model of the data is known a similar approximation can be used to make a ballpark estimate of the size. For each ILF conceptual entity type 35 function points are counted and for each EIF conceptual entity type 15 function points are counted. Deviations of up to 50% have been recorded, so this kind of estimation should only be used as a first estimate. These early sizing techniques based on function point analysis are environment indepentent; In all environments where function point analysis can be applied these early sizing techniques work in the same fashion. Presumably this is the case because of the fact that function point analysis works with distinct sizing classes, but as far as I have been able to trace this assumption is not really supported with evidence. 2.2. Early size estimating using COSMIC-FFP In environments where function point analysis cannot be applied there may be an alternative. For a number of these environments the COSMIC-FFP functional size measurement method has been designed [13]. In the measurement manual of this method two techniques for early size estimation are described [14]. At this moment little practical experience with these techniques has been reported. 2.3. Approximate COSMIC-FFP The approximate technique uses an average value for the size of a functional process. In the very early stages of software development only the number of functional processes is known. To estimate the size of an application the number of functional processes can be multiplied by the average size of a functional process. The measurement manual gives an example, based upon the development of avionics software for a military aircraft, where the average size of a functional process is 8. Experience from the Dutch Rabobank shows that in their development environment the average size of a functional process is 7,2 [15]. This difference suggests that approximate COSMIC-FFP may be environment dependent, although both experiments are based on too small a number of projects to draw conclusions on. 2.4. Refined approximate COSMIC-FFP 1 From our own experience we learnt that this factor is a function of the total size of the information system and that the value of 25 function points for each ILF entity type is only valid if the total size of the information system is less than 10 logical files. If the number of logical files is 10-25 a factor of 28 should be used and if the number of logical files is greater that 25 a factor of 35 should be used. In a later stage of the development process there is enough information about the functional processes to classify them into different categories. The refined approximate technique as described in the Measurement Manual to classify functional processes uses four categories: small e.g. retrieval of information about a single object of interest medium e.g. storage of a single object of interest with some extra checks large e.g. retrieval of information about multiple objects complex To each of these categories average values can be assigned by dividing the number of functional processes into four quarts and computing the average size of a functional process in each of the quarts. During research on the original data it was discovered that there was a discrepancy between the way the measurement manual describes the way in which the quarts should be computed and the way the numbers for the example in the manual were actually computed : Dividing the total size of the software by four and establishing the average size of the quarts, containing the functional processes sorted by size (Method 1) is the method that has been described. Dividing the number of functional processes by four and establishing the average size of the quarts, containing the functional processes sorted by size (Method 2) is the method that has been used. Previous research upon a sample of eleven projects showed that the first method seemed to yield more precise results [16]. Furthermore the different classes could be better distinguished with the first method than with the second. The table below sums up the results from that research: Method 1 Method 2 Quart Average Rabobank Size range Rabobank Average Avionics Average Rabobank Size range Rabobank Small 4,0 cfsu ≤ 5 cfsu 3,9 cfsu 3,6 cfsu ≤ 4 cfsu Medium 6,2 cfsu 5-8 cfsu 6,9 cfsu 4,4 cfsu 4-5 cfsu Large 10,8 cfsu 8-14 cfsu 10,5 cfsu 6,3 cfsu 5-8 cfsu Complex 24,7 cfsu ≥ 14 cfsu 23,7 cfsu 14,9 cfsu ≥ 8 cfsu Table 1 Comparison of results from different environments Since both sources contain just a small number of projects further investigation on the subject was necessary to conclude whether early size estimation with COSMIC-FFP is environment dependent or not. 3. Early size estimating figures in different environments Since the end of 2003 all of Sogeti's internal projects and more and more projects of our customers are being sized with COSMIC-FFP. A number of these projects were selected for an investagition to what extent early size estimating figures are environment dependent. 3.1. Project selection Most of Sogeti's projects are in the domain of business application software [17]. From that domain 47 projects have been selected. From those projects 11 were already used in an earlier investigation [15]. The selected projects can be divided into four sectors: banking 2 clients government 5 clients insurance 4 clients logistics 5 clients 3.2. Figures for approximate COSMIC-FFP In the table below the characteristics of the four sets of projects are presented: Sector Project Functional process number total size average size number average size Banking 26 12.375 cfsu 476 cfsu 1.345 9,2 cfsu Government 8 3.845 cfsu 481 cfsu 838 4,6 cfsu Insurance 6 3.305 cfsu 551 cfsu 342 9,7 cfsu Logistics 7 3.766 cfsu 538 cfsu 321 11,7 cfsu Total 47 23.291 cfsu 496 cfsu 2.846 8,2 cfsu Table 2 Figures for approximate COSMIC-FFP Although the average project size for each sector is of the same order of maginitude, the average size of a functional process for each sector is quite different. The size of an average functional process in the banking sector in this sample (9,2 cfsu) is significantly higher than the average size reported earlier (7,3 cfsu) for the same sector [15]. There is also a significant difference in the average project size of this sample (476 cfsu) in comparison with the earlier sample (339 cfsu). Further investigation into the details of the projects is necessary to find an explanation for these findings. The samples for the non-banking sectors are quite small, but the difference between government projects and the other sectors are quite striking. The fact that the average value for the related sectors banking and insurance are close together supports the idea that there might be a relation between the sector and the average size of a functional process. 3.3. Figures for refined approximate COSMIC-FFP In the tables on the next page the figures for the refined approximate COSMIC-FFP are presented from the projects reported in section 3.2. The first table contains the figures that have been derived using Method 1 (based on size, see section 2.4). The second table contains the figures that have been derived using Method 2 (based on numbers of functional processes). Banking Government Insurance Logistics Quart Method 1 Range Average Range Average Range Average Range Average Small ≤ 7 4,2 ≤ 4 2,3 ≤ 7 5,9 ≤ 9 5,0 Medium 7-10 8,4 4 4,0 7-10 8,2 9-16 11,9 Large 10-31 16,4 4-11 6,9 10-16 12,7 16-39 24,1 Complex ≥ 31 51,6 ≥ 11 20,5 ≥ 16 23,6 ≥ 39 58,8 Table 3 Figures per quart (size) per sector Banking Government Insurance Logistics Quart Method 2 Range Average Range Average Range Average Range Average Small ≤ 4 3,4 ≤ 2 2,0 ≤ 7 5,1 ≤ 4 3,4 Medium 4-6 5,2 2-4 2,7 7 7,0 4-7 5,6 Large 6-8 7,6 4 4,0 8-10 8,8 7-12 9,8 Complex ≥ 8 24,8 ≥ 4 9,6 ≥ 10 17,6 ≥ 13 28,0 Table 4 Figures per quart (numbers of functional processes) per sector As observed earlier, the size ranges for small, medium and large are close together in the figures from Method 2 [16]. The quarts based on the number of functional processes give ranges that cannot be distinguished by an estimator and are therefore not useful for early estimation. The distribution of the functional processes over the quarts is represented graphically in the figures below to illustrate this: Figure 1 Distribution of functional process size per quart (Method 1) 0 20 40 60 80 100 120 140